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^ I Abstract: We show that if X and Y are integers independently and uniformly distributed 
^ ', in the set {!,..., A^}, then the information lost in forming their product (which is given 

S ■ by the equivocation H{X, Y \ X -Y)), is O (log log A^). We also prove two extremal results 
regarding cases in which X and Y are not necessarily independently or uniformly dis- 
^ I tributed. First, we note that the information lost in multiplication can of course be 0. We 
. pi ; show that the condition H{X, Y \ X ■ Y) = implies 21og2 - H{X, Y) = 0(loglog A^). 

Furthermore, if X and Y are independent and uniformly distributed on disjoint sets 
of primes, it is possible to have H{X, Y \ X ■ Y) = with log2 A^ — H{X) and 
log2 N — H{Y) each 0(loglog A^). Second, we show that however X and Y are distributed, 
H{X,Y \ X - Y) = O {log N / \og\og N) . Furthermore, there are distributions (in which X 
and Y are independent and uniformly distributed over sets of numbers having only small 
and distinct prime factors) for which we have H[X, Y \ X ■ Y) = fl{logN/ log log A^). 



1. Introduction 

Let X and Y be random integers. We regard a multiplier as a deterministic channel 
whose input is the pair (X, Y) and whose output is the product X ■ Y. The information 
lost in multiplication is, according to Shannon [S3], the equivocation H{X,Y \ X ■ Y). 
From the definition of conditional entropy, we have 

H{X, Y \ X-Y) = H{X, Y,X-Y)- H{X ■ Y) 

= H{X,Y)-H{X-Y), (1.1) 

where we have used the fact that the channel is deterministic (X • y is determined by X 
and y, so that H(X, Y,X-Y) = H{X, Y)). 

We first consider the case in which X and Y are independent and uniformly distributed 
on the set {1, . . . , A/"}, so that H{X^ Y) = 2 log2 N. We shall show in Section 2 that in this 
case we have 

H{X,Y \X ■Y) = e{\og\ogN). (1.2) 

If X and y have arbitrary (that is, not necessarily independent or uniform) dis- 
tributions on {l,...,Ar}, then it is of course possible that H{X^Y \ X -Y) = 0. We 
may then ask how close H{X, Y) can come to its maximum 2 log2 N, while still achieving 
H{X, y I X • y) = 0. We shall show in Section 3 that H{X, y | X • y) = implies that 

21og2iV-if(X,y) = n(loglogiV). (1.3) 

Furthermore, by taking X and Y to be independent, with distributions concentrated on 
disjoints sets of primes, it is possible to achieve H{X, y | X ■ y) = with log2 — H[X) 
and log2 — H{Y) each 0(loglog A^), so that (1.3) is the best possible bound. 

We shall also consider the distributions of X and Y that maximize the information 
loss. We shall show in Section 4 that for any distributions of X and Y on {1, . . . , N} we 
have 

iy(x,y I x-y) = o(iogAr/ log log AT). (i.4) 

Furthermore, by taking X and Y to be independent, with distributions concentrated on 
integers having only small and distinct prime factors, we can achieve 

H{X, y I X • y) = VL{\og N/ log log AT), 

so that (1.4) is the best possible bound. 
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Results concerning information flow through a multipher have been used by Abelson 
and Andreae [A] and by Brent and Kung [B] to obtain fower bounds involving the area 
and time required for multiplication. Furthermore, the results in Section 4 give a lower 
bound to the number of ancillary lines required by a reversible multiplier (see Fredkin 
and Toffoli [F] for a discussion of reversible computation). This lower bound is achievable 
if multiplication is performed by a single gate; it is an open question whether it can be 
achieved if the multiplier is implemented using standard reversible gates, such as those 
proposed by Fredkin and Toffoli. 

The proofs in this paper draw upon a variety of results from number theory. Many 
of these in turn rely on the prime-number theorem (first proved by Hadamard [HI] and 
independently by de la Vallee Poussin [V]) and its extension to primes in arithmetic pro- 
gressions (first proved by de la Vallee Poussin [V]). While these deep theorems now have 
elementary proofs (due to Selberg [SI, S2] and Erdos [El]), none of our results actually 
depend on theorems of this depth, and thus we shall take care to point out the simplest 
results that support our proofs. 



2. The Uniform Distribution 

Our goal in this section is to establish (1.2). For X and Y independent with the 
uniform distribution, we have 

H{X,Y) = 2\og^N. 

Thus from (1.1) we have 

H{X,Y \X ■Y)^2\og2N -H{X -Y). (2.1) 

Define m{N) by 

m{N) = #{x ■y:l<x<N,l<y<N}. 

We have 

H{X-Y) < log2 m(iV). 

Thus the bound 

H{X, Y\X-Y)^ n(log log N) (2.2) 
is a consequence of (2.1) and the following result. 
Proposition 2.1: For any £ > 0, we have 

m{N) < — (2.3) 

^ ' - (logiV)"-^ ^ ' 

for all sufficiently large A?", where a = 1 — log2(eln2) = 0.08607 
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This result is due to Erdos [E2] , who also proved the matching bound 

iV2 



m(N) > 



(log iV)«+e 

For completeness, we shall give a simple proof of this proposition. 

Proof of Proposition 2.1: Let f{n) denote the number of distinct prime factors in the 
integer n > 1. Let r]^{x) denote the number of integers n in the interval 1 < n < x such 
that f{n) = k. Hardy and Ramanujan [H2] (Lemma B) have shown that there are absolute 
constants L and D such that 

, , Lx (Inlnx + D)''-'^ 

for all /c > 1 and x >2. Apart from an elementary precursor 

T^{x)^0■ ^ 



log a; 

to the prime-number theorem due to Chebyshev [C] , their result relies only on the elemen- 
tary estimates 

p<x 

and 

- = O (log log x) 

p<x ^ 

(in which the sums are over primes p) due to Mertens [M]. We observe that (2.4) implies 

, , Mx (Inlna;)''"^ ,^ , 

for all a; > 2 and 1 < A; < 21og2 In a;, where M = Lexp(2Dlog2 e). 
Fix < 5 < 1/6. Define mi(iV), m2{N) and mz{N) by 

mi{N) = #{(a;,y) : 1 < a; < iV, 1 < y < TV and f{x)+f{y) < (1 + 25) loga IniV}, 
TO2(iV) = : 1 < ^ < and f{z) > (1 + 5) logg In AT} 
and 

ms{N) = 1 < z < and w'^ | z for some w with f{w) > 51og2lnA'"j. 

Then we have 

m{N) < mi{N) + m2{N) + m3(iV). 
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For if z = X - y is not counted by mi{N), then we have f{x) + f{y) > (1 + 25) log2 In AT. If 
in addition z is not counted by m2{N), then we have f{z) < {1 + S) log2 In AT, and thus 

f{gcd{x,y)) > 51og2lniV, 

where gcd(a;, y) denotes the greatest common divisor of x and y. If we now let w be 
the product of the distinct primes dividing gcd{x,y), then we have tu^ | z and f{w) > 
51og2 InN, so that z = x-y is counted by ms{N). Thus it will suffice to show that mi (AT), 
777,2 (A?") and ms{N) each satisfy a bound of the form of that in (2.3). 

For mi{N) we have 

mi(Ar)< Yl ^riN)rj{N) 

i+J<(l+25)log2 InN 

M^N^ ^ (lnlnAr)'=-2 



< 



E 



^ ^ i+j=fe<(l+25)log2lniV ^ ^ ^ 

M^N^ ^ /fc-2\ (lnlnA^)^-2 

- (lnAr)2 ^ U-lJ (/c-2)! 

M^AT^ ^ (21nlnAr)'=-2 

- (lnAr)2 ^ (k-2)\ 

^ ^ i+j=fe<(l+25)log2lnAf ^ ^ 



< 



M^N^ ^ /2elnlnA^^ ''"^ 



E/ ill 111 iv \ . „. 



nniV)2 V /c-2 

^ ^ i+j=fe<(l+25)log2ln7V ^ 

where we have used the definition of mi{N), the bound (2.5), the identity a\/b\ {a — b)\ = 
(l), the inequality (^) < 2°- and the inequality a! > a'^/e'*. 

The summand in (2.6) increases with A; for A; — 2 < 2 In In A/", and decreases thereafter. 
Since k — 2 < (1 + 25) log2 In A?" < 2elnln A?", the largest terms of the sum are those with 
the largest k. There are at most ((1 + 26) log2 In A^) < (21nln A^)2 terms in all, and each 
term is at most 

(2eln2)(^+2'^)i°S2inJV ^ (iniV)(i+2<5)(2-a)_ 
Thus we obtain the bound 

,,,, M2(21nlnAr)2(lnAr)2'5(2-a)iv2 

which is of the form desired, since if 25(2 — a) < e, the factors 

(21nlnA^)2 (In A^)^'^^^-'*) 
in the numerator can be absorbed by the factor (InA?^)^ in the denominator of (2.3). 
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For 777.2 (N) we have 

k>{l+S) logj In AT 



< 



E 



InTV ^ (/e-l)! 



fc>(l+(5)log2lnAr ^ ^ 

where we have used the definition of m2{N), the bound (2.5) the inequahty a! > a"/e". 

The summand in (2.7) increases with A; for A; — 1 < In In A?", and decreases thereafter. 
Since k — 1 > {1 + 5) log2 In AT — 1 > In In A/", the largest terms of the sum are those with 
the smallest k. There are at most 2elog2lnA'" terms with A; — 1 < 2elog2lnA'", and each 
such term is at most 

(eln2)(^+^)'°S2iniv ^ (inAr)(i+'5)(i-«). (2.8) 

Furthermore, all the terms with k — 1 > 2e log2 In A^ are bounded by the terms of a 

geometric progression with ratio 1/2, and thus their sum is bounded by (2.8). Thus we 
obtain the bound 

, , M (1 + 2e log2 In A^) (In iV)'^^!-") A^^ 

^ (h^NT ' 

which is of the form desired, since if 6{1 — a) < e, the factors 

M (1 + 2e log2 In N) (In Ar)'5(i-«) 
in the numerator can be absorbed by the factor (InA?")^ in the denominator of (2.3). 
Finally, for ms{N) we have 

^3 (AT) < E ^ 

/(w)>51og2 IniV 



iV2 



W>Wo 

< — , (2.9) 

Wo 

where wq denotes the smallest integer w such that f{w) > 51og2lnAr. Clearly wq = 
Pi- ■ - Pk is the product of the first k = \6 log2 In N] primes. If N is sufficiently large that 
there are fewer than A;/2 primes that are less than 2"^^^, then wq contains at least A;/2 prime 
factors that are each at least 2^/*^, and thus wq > In AT. The bound (2.9) is therefore also 
of the desired form. This completes the proof of the proposition. □ 
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Next we turn to establishing the upper bound 

H{X,Y \ X -Y) ^0{loglogN). (2.10) 
To do this we use the formula 

H{X,Y\X-Y)= Yl E PAX = x,Y = y]H{X,Y\X.Y = x-y). (2.11) 

l<a;<JV l<y<N 

Using the bound 

H{X, Y \ X -Y = x-y) < log2 i^{{v,w):l<v<N,l<w<Nandv-w = x-y} 

< log2 d{x ■ y) 

(where d{n) denotes the number of divisors of the integer n), we obtain 

H{X,Y\X-Y)< J2 E = x,Y = y]\og^dix-y). (2.12) 

l<a;<iV l<y<N 

For X and Y independent with the uniform distribution, (2.12) becomes 

H{X,Y\X-Y)<^ J2 E log2^(^-y)- (2-13) 

l<a;<iV l<y<N 

Since log2 a is a concave function of a, the average of the logarithm in (2.13) is at most 
the logarithm of the average, and we obtain 

H{X,Y\X.Y)<logJ^ J2 E d(^-y)]- 

\ l<x<Nl<y<N J 

Since d{x • y) < d{x) ■ d{y), we obtain 

H{X,Y\X.Y)<logJ^ J2 E di^)-diy) 

\ l<x<Nl<y<N ^ 

= 21og2 E • (2.14) 

y l<n<N J 

We now use the asymptotic formula 

d{n) = N\nN + 0{N) 

l<n<N 

due to Dirichlet [D2] (which is established simply by estimating the number of lattice 
points in the region bounded by the x-axis, the y-axis and the hyperbola x ■ y = N). 
Substituting this result in (2.14) completes the proof of (2.10), which together with (2.2) 
establishes (1.2). 
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3. Multiplication without Loss of Information 

Our goal in this section is to determine the maximum entropy that X and Y can have 
when H{X, Y \ X -Y) = 0. Let 

W = {{x,y): Fr[X = x,Y = y]> 0} 

denote the support of the distribution of (X, F), and let 

M = {x-y:l<x<N,l<y<N} 

be the range of the multiplication map //:{!,..., A^} x {!,..., N} {!,..., A^^} defined 
by /x(a;, y) = x-y. Then H{X, Y \ X - Y) = implies that /x restricted to W is injective, so 
that #(yy) < #A1 = m{N) and H{X,Y) < loga m(iV). Proposition 2.1 thus shows that 
H{X, Y \ X ■Y) = implies (1.3). 

To show that this result is the best possible, wc let X and Y be independent and 
uniformly distributed over X and 3^, respectively, where X and y are the sets of primes 
that are at most and congruent to 1 and 3, respectively, modulo 4. To show that 
log2 — H{X) and log2 A^ — H{Y) arc each O(loglogA^), it will suffice to show that 

= 7ri^4(A^) and = 'n'3^4{N) arc each 0(A^/logA^). This of course follows from 
the cxtcntion of the prime-number theorem to arithmetic progressions, but we can obtain 
what we need from the following simple result due to Shapiro [S4] (which is an elementary 
quantitative version of the theorem of Dirichlet [Dl] on primes in arithmetic progressions). 
Let a and b be fixed with gcd(a, b) = 1. Then 

p < X 

p = a (mod 6) 

where (p{b) denotes Euler's totient function: the number of a in the range < a < 6 such 
gcd(a, b) = 1. To show that (3.1) implies 

7ra,bix) = Q (-^] , (3.2) 



log a; 

we observe that (3.1) implies that 

In » In A 

— > 

p (j){b) 



Em» m A ^ „ . . 



x/A < p < X 
p = a (mod b) 

for all A > 1, where S is a bound on the magnitude of the 0(1) term in (3.1). Choosing 
A sufficiently large that the right-hand side of (3.3) is strictly positive and observing that 
each term in the sum is at most {Alnx)/x establishes that there must be ri(a;/loga;) terms, 
and thus yields (3.2). 
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4. The Maximum Loss of Information 

Our goal in this section is to determine the maximum possible loss of information in 
multipication. Our starting point is the formula (2.12). Since the average is at most the 
maximum, we have 

H{X, Y \ X -Y) < max max log2 d{x ■ y), 

l<x<N l<y<N 

and since log2 a is an increasing function of a, we obtain 



H{X, Y \ X -Y) < log2 max max d{x ■ y) 

y l<a;<Ar l<y<N 

Using the fact that d{x ■ y) < d{x) ■ d{y) as before, we obtain 



H{X, Y \ X ■Y)<2 log2 (^^mj^^ d{n) j . (4.1) 

Wigert [W] was the first to show that 

/ \ IniV 

log2 max d(n) ~ — — — , (4.2) 
\i<n<N ^ 7 InlniV ^ ^ 

using the prime-number theorem. But Ramanujan [R] has shown that an estimate even 
more precise than (4.2) can be obtained using only the crude bounds 

-W = e(i^) (4.3) 

for the number 7r{x) of primes not exceeding x obtained by Chebyshev [C]. Substituting 
(4.2) into (4.1) yields (1.4). 

To show that this result is the best possible, we let X and Y be independent and 
uniformly distributed on the set V of the 2^ divisors of the product Vk = Pi - ■ - Pk of the 
first k primes, where k is the largest integer such that 

Vk < N. (4.4) 

If we define 'd{x) by 

'd{x) — Inp 

(in which the sum is over primes p), then 

Vk = exp'i?(Pfe), 
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so that (4.4) is equivalent to 

^(pfc) <lniV. 

The bounds 

are equivalent to the bounds (4.3) established by Chebyshev [C]. This implies that 



so that (again using (4.3)) 



From (2.11), we have 



Pfe = e(iogiv), 



/. = e(-i^). (4.5) 

MoglogiV/ ^ ^ 



HiX, Y\X.Y) = ^J2Y1 ^(^' Y\X.Y = x-y). (4.6) 

x€Vy£V 

For x,y & V, let u{x,y) denote the number of primes among pi, . . . ,pk that divide one, 
but not both, of x and y. (This number is also the number of primes that divide the 
square-free part of x ■ y, and thus it depends only on x ■ y.) The random variable {X, Y), 
conditioned on X • y = a; • y, is uniformly distributed over the 2*^*^^'^^ pairs in the set 

U = {{v, w)&VxV:v-w = x- y}, 

so that 

HiX,Y\X-Y = x-y) = \og^#U 

= u(x,y). 

Thus (4.6) yields 

H{X, Y\X.Y)^^J2Y1 "(^' (4.7) 

xevyev 

Since X and Y are each uniformly distributed on the 2^^ divisors of Vk, the divisibility 
of each of X and Y by each of the primes pi, . . . ,pfc is probabilistically equivalent to the 
occurrences of heads among 2k independent flips of an unbiased coin. In particular, each 
of the primes pi, . . . ,pk divides one, but not both, of X and Y with probability 1/2. Thus 
the right-hand side of (4.7) is equal to k/2, and (4.5) yields 

H{X,Y \X-Y) = k/2 

logiV 



log log N 
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This estimate shows that the result (1.4) is the best possible. 
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